The solutions to many real-world problems such as allocating and scheduling resources (e.g., processors, robots, airplanes), digital filter design, planning urban development, configuring a supply chain, designing a manufacturing process, etc., are often obtained by casting the problem to be solved as an optimization problem and then by solving that optimization problem. Typically, a cost/benefit is associated with the optimization problem and solving the optimization problem generally entails minimizing the cost and/or maximizing the benefit. Often, though not always, the cost minimization and/or benefit maximization is performed under one or more constraints. Such optimization problems can be complex and a single processor may take up to several hours or days to solve a particular problem. Therefore, at least partially dividing the optimization problem into parts and distributing the task of solving two or more parts to different processors can result in performance improvement.
In designing distributed, asynchronous algorithms/systems for solving an optimization problem, a common approach is to begin with a non-distributed, single iteration of the optimization problem or with a distributed, synchronous implementation thereof, and to attempt to organize variables so that the optimization algorithm can be distributed across several unsynchronized processing nodes. An important limitation of this strategy is that it does not take into account, a priori, what algorithms might be amenable to a distributed, asynchronous implementation, potentially resulting in architectures of an overly-specific class.